Methods and apparatuses for transferring data

ABSTRACT

The present invention provides several methods and apparatuses for transmitting multimedia data using streaming media protocols such as real-time transfer protocols (RTP) and real-time streaming protocols (RTSP) in a computer network environment. In one exemplary embodiment, a request for RTP data and its associated extension is sent from the caching proxy server to the server. The request may be for one specific type of data or multiple unrelated types of data. The server responds to the request indicating its support for the requested RTP extension data. The caching proxy server determines whether to proceed or terminate the data transmission process based on the response provided by the server. If it is determined to proceed with the data transmission process, the caching proxy informs the server to send the requested and supported RTP data. The server sends the requested data in a variable and extendible header format.

This application is a divisional of co-pending U.S. patent applicationSer. No. 09/603,108, filed on Jun. 22, 2000.

FIELD OF THE INVENTION

The present invention relates to the field of multimedia datatransmission. In particular, the present invention in one exemplaryembodiment relates to multimedia data transmission of real-time transferprotocol (RTP) packets using real time streaming protocol (RTSP) in acomputer network environment.

INTRODUCTION AND BACKGROUND OF THE INVENTION

Methods of transmitting data are commonly known and performed today on aroutine basis to send various multimedia data such as text, graphics,audio, video, images etc. across computer networks situated in variousparts of the world. Generally the transmission process requires bothhardware and software for performing its function. Typically, thehardware includes various types of personal computers and hand heldmultimedia data sending or receiving devices. These devices run underthe control of an operating system and utilize multimedia applicationsoftware programs. As is known in the art, streaming media data is datawhich is transmitted to a receiving computer system and presented(usually after buffering temporarily at the receiving system) and thendiscarded (not stored) at the receiving system.

Currently, data is sent in form of packets from one multimedia device toanother. A large amount of information is required to be sent in areal-time manner in the data packets, which imposes a heavy load on thesystems. Streaming media data, such as Real-Audio data in streamingmedia format specified by Real-Networks, is sent through the Internet isnear real-time manner in many cases.

In one approach, the components involved in data transmission ofstreaming media are known to be a server (which may be referred to asoriginating server), a caching proxy server and a client. Thesecomponents in various combinations communicate with each other fortransmitting data packets in real-time. The communication link thatcurrently exists between the components uses real-time transferprotocols (RTP) and real-time streaming protocols (RTSP) to communicateand send packets to each other. For this approach to work, a cachingproxy server needs to communicate with the system server, receive astream of RTP data packets, and transfer the information containedwithin the RTP data packets to a client. FIG. 1 a shows an example of aprior method in which a caching proxy server receives streaming mediadata and provides this data to a client. In order to perform itsfunction properly and efficiently, the caching proxy server needsseveral pieces of information from the server to be able to cache an RTPstream easily and reliably.

A problem with the current approach is that it is not able to providesome of the key required information such as data packet transmit timeand video packet frame type information that a caching proxy needs to beefficient.

This information allows a caching proxy server to provide smooth packetdelivery to its client by knowing the time an RTP data packet wasintended to be sent, and type of video frame that is being sent withoutknowing the specific payload format. Another problem with the currentapproach is that it is not able to provide multiple pieces of unrelateddata in one delivery to the caching proxy server. Furthermore, packetsfrom the server may be “lost” and never reach the caching proxy server.In addition, there is normally no way to recreate a complete “pristine”copy at the caching proxy server.

Prior art servers communicate RTP information to the caching proxyserver by sending information through a cache-control header. In oneapproach, a cache-control header contains normal header fields. Inanother approach, unrelated to cache control of RTP information, asingle type of additional information has been added to the normalfields in a header extension format without specifying the type ofadditional information. In this approach only a single piece of RTPextension can be added to the normal field of the header and sent at anyone time.

A problem with using this limited, non-extensible approach is that aserver is not able to attach multiple sets of unrelated data at a timeto send to the caching proxy server. Another problem with this approachis that the header extension used in these methods are still not able toprovide all the information a caching proxy server needs to cache astream properly and to transmit the stream properly. Yet another problemwith this approach is that there is no way to identify the particularextension independently of other possible extensions.

SUMMARY OF THE INVENTION

The present invention provides several methods and apparatuses fortransmitting multimedia data using streaming media protocols such asreal-time transfer protocols (RTP) and real-time streaming protocols(RTSP) in a computer network environment. In one exemplary embodiment, arequest for RTP data is sent from the caching proxy server to theserver. The request may be for one specific type of data and its relatedextensions or multiple unrelated types of data and their relatedextensions. The server responds to the request indicating its supportfor the requested RTP data. The caching proxy server determines whetherto proceed or terminate the data transmission process based on theresponse provided by the server. If it is determined to proceed with thedata transmission process, the caching proxy informs the server to sendthe requested and supported RTP data. The server sends the requesteddata in a variable and extendible header format.

In another embodiment, the caching proxy server requests and receivespacket transmit time data and/or packet frame type data from the server.The caching proxy server uses the frame type data to communicate withthe client and supply frames based on client's capacity to handle loadsat given times. Transmit time data is also used by the caching proxy tostore packets locally and deliver these packets at appropriate times tothe client for a smooth packet delivery.

Other features and advantages of the present invention will be apparentfrom the accompanying drawings, and from the detailed description, whichfollows below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and not limitedby the figures of the accompanying drawings in which like referencesindicate similar elements and in which:

FIG. 1 a is a flowchart which shows a method in the prior art fortransferring streaming media data to caching proxy server and then to aclient.

FIG. 1 b illustrates a network of computer systems in which media datamay be exchanged and/or processed, according to one embodiment of thepresent invention.

FIG. 2 illustrates a block diagram of an exemplary digital processingsystem, which may be used in accordance with one embodiment of thepresent invention.

FIG. 3 illustrates one embodiment of a communication method between aserver and a client using RTSP and RTP protocols.

FIG. 4 illustrates another embodiment of a communication method betweena server, caching proxy server and a client.

FIG. 5 illustrates one embodiment of a RTSP, RTP negotiation processbetween a caching proxy and a server.

FIG. 6 illustrates one embodiment of a relationship between the server,caching proxy, and client during a transfer of a Transmit Time (TT)sub-extension to the caching proxy server and its use of TT informationin transmitting streaming data to a client.

FIG. 7 illustrates one embodiment of process that takes place duringtransfer of a transmit time sub-extension between server and cachingproxy server.

FIG. 8 illustrates one embodiment of process that takes place duringtransfer of a frame type sub-extension between server, and caching proxyserver.

FIG. 9 is a flow diagram of one embodiment of an operation to providevarious types of information to a caching proxy in an extensible headerformat.

FIG. 10 illustrates one embodiment of a relationship between the server,caching proxy, and client during a transfer of a Frame Typesub-extension.

FIG. 11 illustrates a block diagram of a machine readable medium whichstores executable computer program instruction for execution by anexemplary caching proxy server, which may be used in accordance with oneembodiment of the present invention.

FIG. 12 illustrates a block diagram of a machine readable medium whichstores executable computer program instruction for execution by anexemplary originating server (server), which may be used in accordancewith one embodiment of the present invention.

FIG. 13 illustrates a block diagram of a machine readable medium whichstores executable computer program instruction for execution by anexemplary client, which may be used in accordance with one embodiment ofthe present invention.

DETAILED DESCRIPTION

A method and system for providing multimedia data transmission usingreal-time transfer protocol (RTP) and real time streaming protocol(RTSP) are described. For purposes of explanation, numerous specificdetails are set forth in order to provide a thorough understanding ofthe present invention. For example, various computer network systemarchitectures and digital processing system architectures are providedfor illustrative purposes rather than to be construed as limitations ofthe present invention. It will be evident, however, to one skilled inthe art that the present invention may be practiced without thesespecific details. In other instances, well-known structures and devicesare shown in block diagram form to facilitate explanation.

FIG. 1 b is a diagram of a network of computer systems in which mediadata may be processed, according to one embodiment of the presentinvention. As shown in FIG. 1 b, a number of client computer system, oneor more of which may represent one implementation of a receiving system,are coupled together through an Internet 122. It will be appreciatedthat the term “Internet” refers to a network of networks. Such networksmay use a variety of protocols for exchange of information, such asTCP/IP, ATM, SNA, SDI, RTP, RTSP etc. The physical connections of theInternet and the protocols and communication procedures of the Internetare well known to those in the art. Access to the Internet 103 istypically provided by Internet service providers (ISPs), such as the ISP124 and the ISP 126, which may also be connected with caching proxyservers 130 and 132. Users on client systems, such as the clientcomputer systems 102, 104, 118, and 120, generally obtain access to theInternet through Internet service providers, such as ISPs 124 and 126,which may also be connected through the internet with caching proxyservers 130 and 132. Access to the Internet may facilitate transfer ofinformation (e.g., email, text files, media files, etc.) between two ormore digital processing systems, such as the client computer systems102, 104, 118, and 120 and/or a streaming media server system 128 whichmay be considered an originating server from which caching proxy serversreceive streaming media data. For example, one or more of the clientcomputer systems 102, 104, 118, and 120 and/or the streaming mediaserver 128 may provide media data (e.g., video and audio, or video, oraudio) to another one or more of the client computer systems 102, 104,118, and 120 and/or the streaming media server 128. Such may be providedin response to a request. As described herein, such media data may betransferred in the system 100 according tracks. Such tracks, in oneembodiment of the invention, may be created according to a specificformat of the streaming media data and/or a specific data communication(e.g., network) protocol(s).

The streaming media server 128 is typically comprised of at least onecomputer system to operate with one or more data communicationprotocols, such as the protocols of the World Wide Web, and as such, istypically coupled to the Internet 122. Optionally, the streaming mediaserver 128 may be part of an ISP which may provide access to theInternet and/or other network for client computer systems. The clientcomputer systems 102, 104, 118, and 120 may each, with appropriate webbrowsing software, access data, such as HTML documents (e.g., Webpages), which may be provided by the streaming media server 128. Suchdata may provide media, such as QuickTime movies or QuickTime streamingmedia data, which may be presented by the client computer systems 102,104, 118, and 120.

The ISP 124 provides Internet connectivity to the client computer system102 via a modem interface 106, which may be considered as part of theclient computer system 102. The client computer system may be aconventional computer system, such as a Macintosh computer, a “network”computer, a handheld/portable computer, a Web TV system, or other typesof digital processing systems (e.g., a cellular telephone having digitalprocessing capabilities). Similarly, the ISP 126 provides Internetconnectivity for the client computer systems 104, 118 and 120, althoughas depicted in FIG. 1 b, such connectivity may vary between variousclient computer systems, such as the client computer systems 102, 104,118, and 120. For example, as shown in FIG. 1 b, the client computersystem 104 is coupled to the ISP 126 through a modem interface 108,while the client computer systems 118 and 120 are part of a Local AreaNetwork (LAN). The interfaces 106 and 108, shown as modems 106 and 108,respectively, in FIG. 1 b, may be an analog modem, an ISDN modem, acable modem, a satellite transmission interface (e.g., “Direct PC”), awireless interface, or other interface for coupling a digital processingsystem, such as a client computer system, to another digital processingsystem. The client computer systems 118 and 120 are coupled to a LAN bus112 through network interfaces 114 and 116, respectively. The networkinterfaces 114 and 116 may be an Ethernet-type, Asynchronous TransferMode (ATM), or other type of network interface. The LAN bus is alsocoupled to a gateway digital processing system 110, which may providefirewall and other Internet-related services for a LAN. The gatewaydigital processing system 110, in turn, is coupled to the ISP 126 toprovide Internet connectivity to the client computer systems 118 and120. The gateway digital processing system 110 may, for example, includea conventional server computer system. Similarly, the streaming mediaserver 128 may, for example, include a conventional server computersystem.

The system 100 may allow one or more of the client computer systems 102,104, 118, and 120 and/or the streaming media server 128 to provide mediadata (e.g., video and audio, or video, or audio) to another one or moreof the client computer systems 102, 104, 118, and 120 and/or thestreaming media server 128. Such data may be provided, for example, inresponse to a request by a receiving system, which may be, for example,one or more of the client computer systems 102, 104, 118, and 120.

FIG. 2 is a block diagram of an exemplary digital processing systemwhich may be used in accordance with one embodiment of the presentinvention. For example, the digital processing system 250 shown in FIG.2 may be used as a client computer system, a streaming media serversystem, a conventional server system, etc. Furthermore, the digitalprocessing system 250 may be used to perform one or more functions of anInternet service provider, such as the ISP 124 or 126. The digitalprocessing system 250 may be interfaced to external systems through amodem or network interface 268. It will be appreciated that the modem ornetwork interface 268 may be considered as part of the digitalprocessing system 250. The modem or network interface 168 may be ananalog modem, an ISDN modem, a cable modem, a token ring interface, asatellite transmission interface, a wireless interface, or otherinterface(s) for providing a data communication link between two or moredigital processing systems.

The digital processing system 250 includes a processor 252, which mayrepresent one or more processors and may include one or moreconventional types of such processors, such as a Motorola PowerPCprocessor, an Intel Pentium (or x86) processor, etc. A memory 255 iscoupled to the processor 252 by a bus 256. The memory 255 may be adynamic random access memory (DRAM) and/or may include static RAM(SRAM). The processor may also be coupled to other types of storageareas/memories (e.g., cache, Flash memory, disk, etc.), which could beconsidered as part of the memory 255 or separate from the memory 255.

The bus 256 further couples the processor 252 to a display controller258, a mass memory 262, the modem or network interface 268, and aninput/output (I/O) controller 264. The mass memory 262 may represent amagnetic, optical, magneto-optical, tape, and/or other type ofmachine-readable medium/device for storing information. For example, themass memory 262 may represent a hard disk, a read-only or writableoptical CD, etc. The display controller 258 controls in a conventionalmanner a display 260, which may represent a cathode ray tube (CRT)display, a liquid crystal display (LCD), a plasma display, or other typeof display device. The I/O controller 264 controls I/O device(s) 266,which may include one or more keyboards, mouse/trackball or otherpointing devices, magnetic and/or optical disk drives, printers,scanners, digital cameras, microphones, etc.

It will be appreciated that the digital processing system 250 representsonly one example of a system, which may have many differentconfigurations and architectures, and which may be employed with thepresent invention. For example, Macintosh and Intel systems often havemultiple busses, such as a peripheral bus, a dedicated cache bus, etc.On the other hand, a network computer, which may be used as a digitalprocessing device of the present invention, may not include, forexample, a hard disk or other mass storage device, but may receiveroutines and/or data from a network connection, such as the modem orinterface 268, to be processed by the processor 252. Similarly, a Web TVsystem, which is known in the art, may be considered to be a digitalprocessing system of the present invention, but such a system may notinclude one or more I/O devices, such as those described above withreference to I/O device(s) 266. Additionally, a portable communicationand data processing system, which may employ a cellular telephone and/orpaging capabilities, may be considered a digital processing system whichmay be used with the present invention.

In the system 250 shown in FIG. 2, the mass memory 262 (and/or thememory 254) may store media (e.g., video, audio, movies, etc.) which maybe processed according the present invention (e.g. by way of tracks).Alternatively, media data may be received by the digital processingsystem 250, for example, via the modem or network interface 268, andstored and/or presented by the display 260 and/or I/O device(s) 266. Inone embodiment, packetized media data may be transmitted across a datacommunication network, such as a LAN and/or the Internet, in accordancewith tracks. On the other hand, the processor 252 may execute one ormore routines to use a file with one or more tracks, or alternatively,to create one or more tracks, to process media (e.g., a pre-packagedmovie, audio file, video file, etc.) for presentation or packetizationaccording to the tracks. Such routines may be stored in the mass memory262, the memory 264, and/or another machine-readable medium accessibleby the digital processing system 250. In one embodiment, the digitalprocessing system 250 may process media data having tracks embeddedtherein. Similarly, such embedded media data may be stored in the massmemory 262, the memory 264, and/or another machine-readable mediumaccessible by the digital processing system 250.

FIG. 3 shows an example of components involved in data transmissionscenario. An originating server 301 and a client 302 are shown ascomponents involved in carrying out transmission of streaming media datausing RTP and RTSP protocols as one embodiment of the present invention.The originating server 301 and the client 302 may communicate directlywith each other or may communicate through an intermediary such as acaching proxy server. In one embodiment, the server 301 and the client302 may be on separate local area networks (LAN). In another embodimentthe server 301 and the client 302 may be connected through a wide areanetwork. There may be either one or several clients 302 that are incommunication with the server 301 directly or indirectly through anintermediary, such as the Internet. The server 301 and client 302 mayinteract with each other for sending various types of streaming mediadata in various formats. In one embodiment, the streaming media data maybe sent in a downstream direction from server 301 to client 302. Inanother embodiment the client 302 may send requests and other streamingmedia data information to server 301.

FIG. 4 shows an example of one embodiment of a communicationrelationship between a client 302, a caching proxy server (CP) 401 andthe originating server 301. There may be several types of connectionsbetween these components, but preferably the client 302 may be incommunication with the caching proxy server 401 through an Internetconnection, and the caching proxy server 401 may be in communicationwith the originating server 301 through an Internet connection

A caching proxy server 401 may be connected through the Internet with asingle client 302 or several clients 302. The caching proxy server 401and its connected clients 302 may be on the same local area network ormay be connected through a wide area network. In one embodiment it ispreferable that the caching proxy server 401 and client 302 or clients302 are connected through a local area network and in close proximity toeach other. An exemplary embodiment of close proximity connection may beconnection in the same company etc. where the connection may utilize ahigh bandwidth interface. The communicational link between the cachingproxy server 401 and client 302 may be of a variety of types such asdirect cable, fiber optic, radio frequency etc. These links may changeand vary based on the need of a particular client 302 and advancementsin technology.

An originating server 301 and a caching proxy server 401 may communicateusing a communicational link such as direct cable, fiber optic, radiofrequency etc. These links may change and vary based on a particularneed and advancements in technology. The cashing proxy 401 may act as anintermediary between the originating server 301 and client 302 totransfer streaming media data and assist in smooth delivery of RTPpackets from server 301 to client 302. In so doing, a caching proxyserver 401 may perform several of its own functions. In one embodimentthe caching proxy 401 functions may be thinning frames, storingstreaming media data locally, and transmitting streaming media data atoffset times to client 302. In another embodiment the caching proxyserver's 401 functions may be negotiating with originating sever 301 forvarious RTP extension associated with various types of streaming mediadata, and receiving or responding to various client 302 requests etc. Inone embodiment, one of the objectives of a caching proxy server 401 isto deliver a pristine and good quality copy of streaming media data tothe client 302 and do so in an efficient and speedy manner.

Typically a client 302 may sent a request directly to the caching proxyserver 401. The caching proxy server 401 may then react to the client302 request and either fetch the requested items from the system serveror responds on its own. Its own response may be from a copy of streamingmedia data which has already been obtained from an originating serverand which has been stored on a storage device controlled by the cachingproxy server (e.g. a local hard disk of the caching proxy server).However the system may also be configured for the client 302 to sendrequests directly to the system server 301 and have the server 301respond back directly to the client 302 or indirectly to the client 302through a caching proxy server 401.

FIG. 5 shows one exemplary method according to an embodiment of thepresent invention. In the operations of FIG. 5, an originating server(e.g. server 301) and a caching proxy server 401 communicate with eachother to assist in smooth transmission of streaming media data. Thiscommunication aids smooth packet delivery in many ways includingallowing the caching proxy server 401 to deliver to the client 302 goodquality streaming media data at a high speed. In addition, thecommunication also aids in assisting and managing client's load byensuring that the client 302 gets a manageable amount of streaming mediadata and no frames are dropped in the process or only less importantframes dropped in the process (through frame thinning).

Initially in operation 501, the caching proxy server requests streamingmedia data from an originating server. The request may be made by askingthe server 301 for “setup” in RTSP for audio or video streaming mediadata. The request may be for one type of streaming media data or severaltypes of streaming media. The request may be for similar or unrelatedtypes of streaming media data. The server 301 receives the request fromthe caching proxy server 401, and the server 301 responds in the mannerdescribed with respect to operation 502 of FIG. 5. The “SETUP” requestin RTSP in operation 501 may be initiated by the caching proxy server401, independently of a client system 302 requesting streaming mediadata or the request in operation 501 may be initiated by a client system302 requesting the streaming media data from the caching proxy server401 which in turn requests the requested streaming media data from theserver 301 (if the caching proxy server 401 does not already have therequested streaming media data stored under its control, such as a localhard disk of the caching proxy server 401). The caching proxy server 401may also log client's IP address for subsequent communication in thecase where a client initiated the request.

The caching proxy server 401 and originating server 301 may establish acommunication process in which the caching proxy server 401 and theoriginating server 301 may engage in a negotiation process 502 forcommunicating back and forth in order to aid a smooth streaming mediadata packet transmission. As shown in operation 501, the caching proxyserver 401 may communicate with the originating server 301 and request(e.g. by specifying names of RTP extensions) a set of RTP extensionsassociated with the streaming media data to be sent to the caching proxyserver 401. The set of extensions requested to the server 301 may be thesame as the set of requests sent to the caching proxy server 401 fromthe client 302 (in those cases where the client specifies RTPextensions, such as security extensions, for its use).

The server 301 receives the request for RTP extensions from the cachingproxy server 401. The server 301 may then run its internal processes todetermine whether the server 301 supports the requested RTP extensions.The outcome of this determination may be that the server 301 supportssome but not all the requested RTP extensions, or that the server 301supports none of the requested RTP extensions, or that the server 301supports all of the requested RTP extensions. The server 301 may respondin operation 502 to the caching proxy server 401 by informing thecaching proxy server 401 of the server's 301 supported RTP extensions.The server 301 may choose to respond 502 by indicating only thesupported RTP extensions or may respond by indicating both the supportedand unsupported RTP extensions, or the server 301 may not respond at allindicating no support for requested extensions. In one embodiment theresponse may be in an echo form or any several other forms. In one echoform of the invention, the server transmits the names of the requestedRTP extensions and an associated code for each named extension.

The caching proxy server 401 receives a response from the server 301indicating the supported RTP extensions or both the supported andunsupported RTP extensions. The caching proxy server 401 may check tosee if a response has been sent for all the RTP extensions it hadearlier requested. Caching proxy server 401 may have received none, one,some, or all responses to the requested RTP extensions. Caching proxyserver 401 may evaluate further to check if any of the server 301unsupported RTP extensions are required for streaming media datatransmitting process. Required RTP extensions may be defined as RTPextensions that are necessary for carrying on a particular datatransmission operation such as frame thinning etc at the caching proxyserver 401. As shown in FIG. 5, operations 501 and 502 relate to setupand negotiation for an audio track while operations 503 and 504 relateto similar setup and negotiation for a video/image track.

In one embodiment, the caching proxy server 401 may request multiplesets of RTP extensions at a time from the server 301. If the RTPextensions requested are required and unsupported by the server 301,then caching proxy server 401 may decide to terminate the negotiationprocess. It may also be the case that some of the extensions aresupported and some are not. In such a situation, if the unsupportedextensions are not required for the data transmission process thencaching proxy server 401 may decide to proceed further and receive thesupported extensions and the associated streaming media data. In anotherembodiment the caching proxy 402 may not receive a response for any ofthe RTP extensions requested. In such a case the caching proxy 402 maychoose to terminate the negotiation process with the server 301.

If the caching proxy server 402 decides not to terminate the negotiationprocess and to request the supported RTP extensions and streaming mediadata, it may send a request to the server 301 to send the streamingmedia data and the associated supported RTP extensions in operation 504.In the example of FIG. 5, this request for the streaming media data andthe associated RTP extensions occurs when the caching proxy server 401sends a “PLAY” command in the RTSP protocol.

The server 301 in operation 505, responds to the “PLAY” command bysending the streaming media data and by sending the requested andsupported RTP extensions, which is associated with the streaming mediadata, to the caching proxy server 401 in a extended header format. Thisheader may contain one, two or three similar or unrelated RTPextensions.

Upon receiving the streaming media data and receiving RTP extensionsfrom the server 301, the caching proxy 401 may store the streaming mediadata and the RTP extensions in a storing facility 601 (e.g. a storagedevice controlled by the caching proxy sever 401, such as a local harddisk of the server 401) and terminate the transmission process with theserver 301. The caching proxy 401 may again reinitiate the negotiationprocess and repeat all the back and forth if another request forstreaming media is submitted by the client 302. This request may besimilar or completely different from prior requests. Some of theextensions that may be requested by the cashing proxy sever 401 may be atransmit time sub-extension denoted by symbol “trti”, or frame typesub-extension denoted by symbol “ftry”, or packet position sub-extensiondenoted by symbol “papo”. Other extensions may also be requested (e.g.an extension which is used by the client 302 or server 401 to maintain asecure or encrypted or authenticated communication between client 302and server 401).

For example, in one cycle of its operation a caching proxy server 401may ask for three separate RTP sub-extensions one of which may be frametype sub-extension denoted by symbol “frty” (used in frame thinning bycaching proxy server 401 as described below), the other may be transmittype sub-extension denoted by “trti” (used by the caching proxy server401 as described below), and the last may be packet positionsub-extension denoted by “papo” (which may be used to retrieve lost ormissing packets). Let us also assume for the illustration of thisexample that “frty” sub-extension is required for the streaming mediadata transmission process. “Frty” may be denoted as a requiredsub-extension due to several reasons. One of the reasons may be that theclient 302 cannot receive or process the data at a high data rate (andso frame thinning is required) and “frty” sub-extension will assist thedata transmission process between a caching proxy server and the client302 by allowing the caching proxy sever to perform frame thinning andtherefore may be “necessary”.

The caching proxy 402 may receive the request and communicate with theserver 301 by sending a single request to the server 301 asking for bothsub-extensions. Let us assume further for the illustration of thisexample that the server 301 can only support one of the two RTPextensions. The server 301 may then send a response back to the cachingproxy server indicating which sub-extension is supported.

If the supported sub-extension happens to be only “trti”, or “papo” orboth but not “frty” then the caching proxy 402 will terminate thenegotiation process between the caching proxy 402 and the server 301.This is because “frty” was a required extension to the data transmissionprocess and since it is not supported by the server 301, the cachingproxy 402 may not proceed further. If however, the supportedsub-extension happens to be only “frty”, or frty and papo, or frty andtrti, or frty, papo and trti, then the caching proxy server 401 mayproceed further with the transmission process. The caching proxy server401 in this instance may choose not to terminate the process since therequired sub-extension frty is present in the response as supported bythe server 301.

FIG. 6 shows an example of a method for transmitting packet transmittime data which may be used with various embodiments of the presentinvention. The server 301 is connected with the caching proxy server 401by way of a standard communication carrying devices such as fiber opticwire link, radio frequency communication, cable wire etc. A personhaving ordinary skill in the art will appreciate that anyone-communication device is not essential for the data transferoperation in accordance with this invention and that thesecommunications devices are interchangeable. It must be clear that it isimportant for the communication devices to allow communication in bothdirections i.e. from server 301 to caching proxy 402 or from cachingproxy 402 to server 301.

The communication between a caching proxy server 401 and the originatingserver 301 may be a direct communication relationship or there may alsobe other devices such as routers in the Internet acting asintermediaries to assist in streaming media data transfer. Typically, acaching proxy server 401 is located in closer proximity to the client302 than the originating server 301. This close proximity may be withina company, or on a designed local area network (LAN), or in the samegeographic region, whereas typically caching proxy server and originalsystem server 301 are further apart.

The caching proxy server 401 may have a storage facility 601 to storestreaming media data 603 and/or the associated RTP extensions 602. Thestorage facility 601 may be a local to the caching proxy server 401 oron an offsite from the caching proxy server 401 but in either case thestorage is controlled by the caching proxy server 401. The caching proxyserver 401 may have a link established to store data received from theserver 301 for a periods of time in the storage facility 601, and thenbe able to retrieve the stored data at a later time for sending toclient 302. In the example of FIG. 6, the streaming media data 603 andits associated RTP extension (transmit time in this case) are storedtogether on a storage device 601. Groups of streaming media data (e.g. apacket or a set of packets) are associated with a correspondingdesignation of a transmit time so that each group has a transmit timewhich specifies when to transmit the particular group. It will beappreciated that the streaming media data and the associated RTPextension may be stored separately (but still be associated—e.g. packetNo. xxx to be transmitted at time ABC, packet No. xxy is to betransmitted at time ABD, etc.)

In the example of FIG. 6, the streaming media data is received by theserver 401 and the caching proxy server 401 receives the transmit timedata from server 301 and stores it in the storing facility 601. Transmittime data may be associated with each track of streaming media data. Forexample, in one instance the transmit time at 0 sec 602 may beassociated with corresponding streaming media data 603. In operation, inthis exemplary embodiment, the streaming media data 0 will be sent to aclient at transmit time 0.

FIG. 7 shows one exemplary method for using transmit time as an RTPextension according to an embodiment of the present invention. Inoperation the method suggested in FIG. 7 may utilize the systemarchitecture as suggested in one of the embodiments of the presentinvention shown in FIG. 6.

In one example of the method of FIG. 7, a caching proxy server 401receives a request from client 302 for streaming media data and thenrequests an RTP extension which specifies transmit time information andrequests the server 301 to send transmit time sub-extension RTP data 701and associated streaming media data. Operation 701 shows the cachingproxy server's request for streaming media data and transmit time whichresults from this request. The server receives the request in operation702 as shown in FIG. 7. It may also be the case that a caching proxyserver 401 already had received the requested streaming media data andits associated transmit time information from the server 301 and hasstored the streaming media data and associated RTP extensions at astoring facility 601. If such, then the caching proxy server 401 maystart responding to clients 302 request without communicating with theoriginating server 301 thereby shipping to operations 707 and 708 ofFIG. 7.

Assuming for illustration of this example that the original server 301supports the transmit time information, server 301 will respond back tocaching proxy server indicating its support of the requestedsub-extension in operation 703. If however the transmit timesub-extension is not supported by the original server 301, theoriginating server 301 may or may not respond back to the caching proxyserver 401 indicating its support for the requested sub-extension asshown in operation 709. In the event of an unsupported sub-extension,the caching proxy 402 may terminate the negotiation process as shown inoperation 710 with the server 301 and would typically inform the client302 of the inability to provide streaming media data. In so doing, thecaching proxy server 401 may first evaluate whether the missing transmittime information is required for running its processes. If the result ofthe determination is that transmit time information in this particularexample is a required element, then the caching proxy server may decidewhether to proceed or terminate the transmission process.

The server 301 in operation 704 sends the transmit time RTP data in anextended header format according to the RTP protocol to the cachingproxy server. The header may consist of the normal header fields, thesub-extension character name and a sub-extension ID 704. Thesub-extension character name for a transmit time data may be a4-character code denoted by “trti”. This code may uniquely identify anddescribe the content of the sub-extension as being transit time data.The sub-extension ID may identify the sub-extension in the RTP packet.

A transmit time sub-extension may consist of a single 64-bit unsignedinteger representing the recommended transmission time of the RTP packetin milliseconds as shown in operation 704. The transmit time may beoffset from one another from the start of a media presentation. Forexample in one sub-cycle of operation, a session description protocoldocument for a uniform resource locator (URL) may include a range of0-729.45 seconds. The client 302 may make a PLAY request 706 for thevideo, audio, text, graphics, and images etc. type data.

The caching proxy server 401 may receive the RTP data packet associatedwith streaming media data with the transmit time sub-extension as shownin more detail in FIG. 6. The caching proxy server 401 may then storethe RTP transmit time data locally as shown in FIG. 6. The caching proxyserver 401 may then strip off the header ID in operation 705 and sendstreaming media data associated with each track, in operation 707, oftransmit time individually at offset times to the client 302 allowingthe client 302 to carry on PLAY operation 708. An advantage of knowingand storing transit time at offsets locally at the caching proxy server,it may now be possible for the caching proxy server 401 to selectivelyre-transmit data at different intervals to the client 302 or respond toclients request to send data corresponding to any particular time slot.

FIG. 8 shows one exemplary method for a stream thinning process by acaching proxy server according to an embodiment of the presentinvention. In operation a client 302 and caching proxy server 301communicate with each other to assist in sending and receiving streamingmedia data and assisting in traffic flow control to the client 302. In amethod according to FIG. 8, a client 302 communicates with the cachingproxy server 401 and indicates that it is overloaded or the cachingproxy server 401 detects that the client is overloaded. As part of thiscommunication, the caching proxy server 401 ensures that the client 302does not get an amount of data that exceeds its data handling capacity.Caching proxy server also prevents at least selected frame being“dropped” or missing as a result of an overloaded client 302.

A principle behind FIG. 8 is that an overloaded client 302 may notifythe caching proxy server that it has reached its capacity for receivingRTP data (e.g. streaming media data). The client 302 may have beenoverloaded due to several reasons including that a caching proxy serveris sending RTP data very quickly and the client 302 is having difficultyreceiving data at such a fast pace. The client 302 may inform thecaching proxy server to stop sending streaming media data altogether, orto send data at a slower pace. The client 302 may also inform thecaching proxy server to send only selected order of frames and not sendany low order frames. The caching proxy server 401 will use the frametype data to determine which frames to transit to client 302; typically,higher priority frames are transmitted while lower priority frames arenot transmitted.

A method of FIG. 8 begins in operation 801 in which a caching proxyserver 402 may communicate with the originating server 301 and requestthe server 301 for streaming media data and its associated frame typeinformation. The frame type identifies various types of data (e.g.frames) in streaming media data which allows “thinning” which may bedefined as reducing frames, sending frames at a slower pace, or notsending certain frames at all. It will be appreciated that thinningapples to various types of data and that “frames” may be considered tobe such various types of data. The server 301 may receive the request inoperation 802 and may respond in operation 803 to the caching proxyserver 401 indicating whether the server 301 supports the requestedframe type streaming media data. If the server 301 supports this, theserver's 301 response in operation 803 includes sending the associatedRTP frame type sub-extension in a format described in block 804 alongwith an identifier code corresponding to the frame type extensionrequested by name in operation 801.

If the server 301 does not support frame type sub-extension then thecaching proxy server may terminate in operation 807 and 808 thecommunication with server 301. The server 301 may indicate that it doesnot support the requested frame type streaming media data by eitherresponding or not sending any response to the Caching Proxy server 401which would also indicate no support of the requested RTP extension forthe streaming media data. However, if the server 301 supports the frametype sub-extension, the caching proxy 402 may inform the server 301 tosend the streaming media associated with the frame type information. Inone embodiment, the server 301 may send the supported streaming mediadata sub-extensions without any further requests from the caching proxyserver 401. In another embodiment, the server 301 may wait for further afurther request from the caching proxy server 401 to send the supportedstreaming media data sub-extensions.

The server 301 may then send the RTP sub-extension in an extended headerformat. The frame type sub-extension may consist of a single 16-bitunsigned integer value with several well-known values representingdifferent frame types. The well-known values may be “1” for a key frame,“2” for a p-frame, or “3” for a b-frame where key frame maybe of thehighest order and most importance, b-frame of the lowest order and leastimportance, and b-frame somewhere between key frame and b-frame in termsof importance. There may also be other frames that may be added to thisformat.

The caching proxy server 401 may then store the streaming media data andits associated frame type sub-extension in its storing device 601 afterreceiving them from the originating server 301. This is shown inoperation 805 of FIG. 8. The caching proxy server 401 may then enterinto a negotiating process with the client 302 in evaluating theclient's capability at the time to handle streaming media data traffic809. Based upon the result of the negotiation process 809, the cachingproxy server 401 may thin frames (sending only selected, predeterminedframes) and send streaming media data associated with selected frames806 to the client 302.

For example, in one cycle of operation a client 302 may inform thecaching proxy server that it is overloaded. The client 302 may informthe caching proxy server 401 to stop sending frames altogether or tolower the bit rate if the transmission falls behind. In the case oflowering the bit rate and slowing down, the caching proxy server 401 maystop sending the lowest order frames of the streaming media data, theb-frame to the client 302. The caching proxy server 401 and the client302 may communicate further to evaluate if the client 302 is stilloverloaded. In one embodiment, if the client 302 is capable of handlingthe load after thinning of the b-frame then the caching proxy server maysend the client 302 key-frames and p-frames. However if the client 302is still overloaded then the caching proxy server 401 may further reducethe data traffic to the client 302 and stop sending p-frames. Thecaching proxy server 401 may further evaluate client's 302 data handlingcapability and determine if any more frame thinning is necessary toreduce load on client 302. In another embodiment the client 302 maydirectly specify to the caching proxy server 401, which frames to sendand which frames not to send until a subsequent request is sent to thecaching proxy server 401 to change sending considerations.

After a client 302 retains its capability to cache frames, the cachingproxy server 401 may again start sending the lower order frames to theclient 302. It may again send all the frames at a high speed or send theframes according to requests received by the client 302. In the eventthat the client 302 gets overloaded again, the caching proxy server 401may repeat the thinning process until the client 302 is able to handlecaching data again. FIG. 10 shows an example of how a caching proxyserver 401 receives streaming media data and its associated frame type(FT) RTP extension data from an originating server 301 and stores thestreaming media data and associated frame type extension data on astorage device (e.g. a local hard disk of the caching proxy server 401)and then uses the frame type data to selectively thin frames of thestreaming media data which is being transmitted to a client 302.

Communication between a caching proxy server 401 and originating server301 or caching proxy server 401 and client 302 is carried on usingreal-time transfer protocol (RTP) and real-time streaming protocol(RTSP) for sending/receiving streaming media data. An originating server301 sends streaming media data packets in a streaming media format usingRTP to a caching proxy server 401 whenever a transmission of streamingmedia data occurs. One of the embodiments of the present invention is tobe able to modify the current existing RTP headers by being able toexpand the header with sub-extensions and also be able to make theheader format variable. Expansion of the header is useful because acaching proxy server 401 may need several pieces of information alongwith a RTP packet that will aid in providing a good quality streamingmedia data packet and smooth delivery to the client 302. The extrainformation that may be needed can be provided by attaching it to theexisting header by being able to expand the header field. It should alsobe clear that variability of the extended header is important becausethe extra pieces of information needed by the caching proxy 402 may varyeach time. To accommodate for this variation, the extended header mayhave the capability to change and provide various types of informationas needed by the caching proxy server 401.

In accordance with one embodiment of the invention, in operation, anextended header consists of a normal header fields. A person havingordinary skill in the art is aware of the various header fields that arenormally used in operation. The normal header fields are immediatelyfollowed by header extension fields. The extension field consists ofseveral sub-extensions. There may be several header sub-extensions thatare unrelated to each other and may vary per request of the cachingproxy server 401. The sub-extensions may have an extension type of “se”.The RTP extension length may be the total length of all thesub-extensions and may be defined in 32-bit words thereby being in fullcompliance with the RTP protocol.

The “se” sub-extension format may be such that a sub-extension IDimmediately follows the normal RTP header field. The ID may identify thesub-extension within the RTP packet. This ID may be a one octet IDgenerated by the server 301 for each individual named RTP sub-extension.Each sub-extension may also have its unique name that is defined by afour-character name code. This name code uniquely identifies anddescribes the type of data in each sub-extension. For example, the fourcharacter name code for a transmit time sub-extension may be “trti”,frame type sub-extension may be “frty” and packet position sub-extensionmaybe “papo”. This name code is associated with the one octet ID(generated by the server 301) so that the caching proxy server 401 canidentify, form the octet ID the appropriate RTP extension data when itreceives streaming media data.

In one embodiment of the present invention, the unique name may be“frty” associated with streaming media data for frame type information.The unique name “frty” may also have an unsigned integer associated witheach different type of frame. In one embodiment the unsigned integer maybe “1” for a key-frame, “2” for a p-frame, and “3” for a b-frame. A usermay also add any additional frames in the future as need and technologyadvances and may use this header format without any need for muchmodification.

In another embodiment of the present invention, the unique name may be“trti” associated with streaming media data for transmit time typeinformation.

In another embodiment of the present invention, the unique name may be“papo” associated with streaming media data for packet position typeinformation.

FIG. 9 shows an exemplary method of several aspects of the presentinvention. In a portion 901, a caching proxy server 401 requestsstreaming media data from an originating server 301 and also requests byname one or more RTP extensions. This request is made using the RTSPprotocol. In operation 903, the server typically responds back (e.g. ofa response would be an echo) a response to the caching proxy server 401indicating its support for the requested RTP extensions. The server 301also transmits to the caching proxy server 401 an identifier, such as anumber code which corresponds to each name of the requested RTPextensions. Typically, the caching proxy server 401 will use the numbercode later in identifying received extended RTP data. The number codeallows the caching proxy server 401 to identify the various types of RTPextension data in the streaming media which it receives as the server301 may not use the name to designate the RTP extension type. Inoperation 905, the caching proxy server 401 receives the server's 301response and then in operation 907, the CP server 401 determines whetherthe server 301 responded to all of the requested RTP extensions.

If the server 301 did not respond to all requested RTP extensions, thenprocessing proceeds to operation 909, followed by operation 911 in whichit is determined whether any of the missing RTP extensions are criticalto the caching proxy server's 401 processing. If they are not critical,then processing proceeds to operation 921. If they are critical, thenthe caching proxy server 401 determines in operation 913 whether or notto terminate the operation/communication with the originating server301. As shown in operations 915 or 917, the caching proxy server 401 mayterminate operations/communications with the server 301 for thisparticular streaming media data which was requested or they proceed toreceive the streaming media and whatever supported extensions can beprovided.

In operation 921, the CP server 401 requests the originating server 301to send the requested streaming media data and its associated RTPextensions. In one embodiment, the CP server 401 transmits a “PLAY”request using RTSP, and this causes the server 301 to respond inoperation 923 by transmitting the streaming media data and theassociated RTP extensions. In operation 925, the CP server 401 storesthe streaming media data received from the server 301 and also storesthe associated RTP extension data. In operation 927, the CP server 401may remove certain RTP extension data from the streaming media file,such as the transmit time or the frame type data. This is done in orderto avoid sending the transmit time or the frame type information to theclient 302 which requests streaming media data. The RTP extension data,which is removed from the streaming media data, is stored separately butassociated with the streaming media data. For example, transmit timesfor various packets are stored separately from the packets, but theassociation existing in the data received from the server 301 betweenthe transmit time and the corresponding packets is maintained even whenthe transmit times are stored separately so that the caching proxyserver 401 may determine the appropriate transmit time for each of thepackets in the streaming media data. In operation 929, the caching proxyserver 401 evaluates a client's 302 request for streaming media data andresponds accordingly. It will be appreciated that a client 302 willnegotiate for streaming media data using the RTSP protocol and the CPserver 401 will respond with the streaming media data by transmittingthe data to the client 302. In addition, the client 302 may requestframe thinning. Further, the caching proxy server 401 may use thetransmit times to determine when to transmit to various packets in thestreaming media data to the client 302.

FIG. 11 shows one type of exemplary machine readable media (e.g. RAM orhard disk or combination thereof) for storing executable computerprogram instructions for a caching proxy server 401 that may be used inaccordance with the present invention. The caching proxy server 401typically will have its own operating system (OS) software 1101. Thissoftware 1101 may be the Macintosh OS. Or Windows NT or Unix, or otherwell known operating systems

The control software 1102 is for transmitting or receiving streamingmedia data using, for example RTP and RTSP protocols. The software 1102is normally able to retrieve or send various types of streaming mediadata packets and direct commands for storing the received media in astoring facility 601. Thus software 1102 performs the negotiationprocess with an originating server 301 and receives streaming mediadata, and its associated RTP extensions and causes the streaming mediadata and its associated RTP extensions to be stored on a storage devicecontrolled by caching proxy server 401. FIG. 11 shows the storage of twostreaming media data files 1103 and 1104.

Streaming media data file 1103 may contain streaming media data 1 instreaming media format 1105, transmit time associated with streamingmedia 1 (1106), and frame type associated with streaming media 1 (1107).In one embodiment, the operating system 1101 and control software 1102may have the capability to separate streaming media data in packet 1from other packets and store it separately in a storing facility 601 andto extract the RTP extensions (e.g. Transmit Time data or Frame Typedata) from the stored streaming media packets and store these separatelyso that these packets do not include the RTP extensions.

Streaming media data file of 1104 may contain streaming media data 2 instreaming media format 1108, transmit time associated with streamingmedia data 2 (1109), and frame type associated with streaming media 2(1110).

The streaming media data 1105 and 1108 will usually not be in the sameoriginal format as the media data was at the originating server 301. Thestreaming media data 1105 and 1108 may however be a full “pristine” copyof the original media data, because the “papo” extension may be used bythe caching proxy server 401 to search for any missing packets in thestreaming media data 1105 and 1108 and to request (again) these packetsfrom the originating server.

FIG. 12 shows one type of exemplary machine readable media (e.g. RAM orhard disk or combination thereof) for storing executable computerprogram instructions for an originating server 301 that may be used inaccordance with the present invention. The server 301 will typicallyhave its own operating system 1201.

The control software 1202 is for transmitting streaming media data to acaching proxy server 401 or to a client 302 using the RTP and RTSPprotocols and the RTP extensions of the invention. Further, software1202 receives requests from a client 302 or a caching proxy server 401for streaming media and negotiates with a caching proxy server 401 forvarious types of streaming media data and associated RTP extensions, andresponds to various requests by caching proxy servers 401 or clients302.

Software 1204 converts original media data 1203, which is usually not ina packet format, to a streaming media data format (e.g. packet format)for transmitting to caching proxy sever 401 or client 302. Whenconverted, the converted streaming media data is a representation of theoriginal media data 1203 that has a different format than the format ofthe original media data 1203.

The software 1206 creates RTP extension headers associated with varioustypes of streaming media data. The system may assign various ID namesand codes 1205 associated with various RTP extensions to various typesof streaming media data before its sent to a caching proxy 401 or aclient 301. The software 1206, in conjunction with software 1202,performs the negotiation process with a caching proxy server 401 (or, insome cases where the client asks for an RTP extension, such as asecurity or encryption or authentication extension, the client) totransmit RTP extension data for an associated streaming media data andalso performs the transmission process of transmitting streaming mediadata with its associated RTP extension.

FIG. 13 shows one type of exemplary machine readable media (e.g. RAM orhard disk or combination thereof) for storing executable computerprogram instructions for a client server 302 that may be used inaccordance with the present invention. The client server 302 willtypically have its own operating system 1301 such as a Macintosh OS, orWindows NT, or Unix, or other well-known operating systems. The client'smedia may also include Web Browser software 1303 such as Netscape'sNavigator or Microsoft's Internet Explorer.

The streaming media data player software 1302 is for receiving andplaying streaming media data transmitted to the client using the RTPprotocol. The streaming media data player software 1302 may be Quicktimesoftware from Apple computer or the Real Player from Real Networks. Thestreaming media data player software 1302 is typically able to sendrequests to a caching proxy server 401 or a server 301 for variousdifferent types of streaming media data and to receive and present (e.g.display images and produce sound) a representation of streaming mediadata.

In yet another embodiment the streaming media data player software 1302may be able to communicate and negotiate with a caching proxy server 401in order to regulate incoming data traffic to handle its load better(e.g. the software 1302 may ask a CP server 401 to perform framethinning).

In the foregoing specification, the invention has been described withreference to specific exemplary embodiments thereof. It will, however,be evident that various modifications and changes may be made theretowithout departing from broader spirit and scope of the invention as setforth in the appended claims. The specification and drawings are,accordingly, to be regarded in an illustrative rather a restrictivesense.

1.-20. (canceled)
 21. A method for extending an RTP header comprising:adding a first RTP sub-extension ID to an RTP header; defining a lengthof said first RTP sub-extension by providing a sub-extension length;providing data corresponding to the RTP sub-extension ID within saidlength defined for said first RTP sub-extension; and having subsequentRTP sub-extensions following the first RTP sub-extension.
 22. A methodof claim 21, wherein the length of the RTP sub-extension being definedby a whole number of 32 bit words.
 23. A method of claim 21, wherein thefirst RTP sub-extension immediately following the RTP header.
 24. Amethod of claim 21, wherein RTP sub-extension length is immediatelyfollowed by RTP sub-extension data and immediately preceded by RTPsub-extension ID.
 25. A method of claim 21, wherein the RTPsub-extension contains transmit time information of each RTP packet. 26.A method of claim 21, wherein the RTP sub-extension contains persistentID information.
 27. A method of claim 21, wherein the RTP sub-extensioncontains Frame Type information.
 28. A method of claim 27, wherein theframe type being a 16-bit unsigned integer value representing adifferent frame for each value.
 29. A method of claim 28, whereinunsigned integer and frame type comprising: assigning an integer valueof “0” to an unknown frame type; an integer value of “1” to a key frametype; an integer value of “2” to a p-frame type; and an integer value of“3” to a b-frame type.
 30. A method of claim 29, wherein said key-framebeing most important in priority than any other frames.
 31. A method ofclaim 29, wherein said p-frame being less important in priority thankey-frame, and more important in priority than b-frame.
 32. A method ofclaim 29, wherein said b-frame being less important in priority thanp-frame.
 33. A method of claim 29, wherein said b-frame being lessimportant in priority than key-frame.
 34. A method of claim 29, whereinsaid unknown-frame being either more or less important in priority thankey-frame, p-frame and b-frame.
 35. A method of claim 29, wherein saidkey-frame being more important in priority than p-frames, b-frames, andany other frames.
 36. A method of negotiating for various types ofstreaming media data by the server comprising: receiving a request forone or more types of streaming media data from a caching proxy server ora client, said request including a request for data associated with saidstreaming media data, said request including an identifier whichrepresents one of several possible types of data associated with saidstreaming media data; determining if requested types of streaming mediadata are supported by the server; and responding to the request with aresponse indicating the capability of the server to support the request.37. A method of claim 36, further comprising receiving a request to sendsupported RTP extensions to the caching proxy or the client.
 38. Amethod claim 37, further comprising responding to request to send andsending all the supported and requested extensions.
 39. A method ofclaim 36, further comprising receiving a command terminating thenegotiation process.
 40. A method of negotiating for various types ofstreaming media data by the caching proxy server comprising: sending arequest for one or more types of related or unrelated streaming mediadata to a server, said request including a request for data associatedwith said streaming media data, said request including an identifierwhich represents one of several possible types of data associated withsaid streaming media data; receiving a response to each requested typeof streaming media data; and deciding whether to proceed or terminatenegotiation process associated with streaming media data.
 41. A methodof claim 40, wherein deciding comprises: determining if any requestedtype of streaming media data is not supported by server; checking ifunsupported type of streaming media data is essential to caching proxyserver operations; and sending an execution command to server.
 42. Amethod of claim 40, wherein determining supported types of streamingmedia data being performed by checking if a response in a form of anecho or otherwise has been sent for requested type of streaming mediadata.
 43. A method of claim 41, wherein execution command being sentbased on results of checking if unsupported type of streaming media datais essential to caching proxy server operations.
 44. A method of 40,wherein decision being to terminate negotiation process.
 45. A method of40, wherein decision being to proceed with negotiation process andrequesting the server to send remaining supported type of streamingmedia data.
 46. A method of frame thinning by caching proxy servercomprising: receiving a message from a client, said message indicating aneed to thin streaming media data being sent to said client; evaluatingpriority of streaming media data; and sending only selected streamingmedia data.
 47. A method of claim 46, wherein said evaluatingcomprising: naming unsigned integers to frame types; assigning aninteger value of “0” to an unknown frame type; an integer value of “1”to a key frame type; an integer value of “2” to a p-frame type; and aninteger value of “3” to a b-frame type.
 48. A method of claim 47,wherein said key-frame being most important in priority than any otherframes.
 49. A method of claim 47, wherein said p-frame being lessimportant in priority than key-frame, and more important in prioritythan b-frame.
 50. A method of claim 47, wherein said b-frame being lessimportant in priority than p-frame.
 51. A method of claim 47, whereinsaid b-frame being less important in priority than key-frame.
 52. Amethod of claim 47, wherein said unknown-frame being either more or lessimportant in priority than key-frame, p-frame and b-frame.
 53. A methodof claim 47, wherein said key-frame being more important in prioritythan p-frames, b-frames, and any other frames.
 54. A method of claim 46,further comprising: receiving a second message from a client to furtherthin streaming media data; processing message and eliminating moreselected streaming media data and sending streaming media data of higherpriority.
 55. A method of claim 54, wherein said selected streamingmedia frame being eliminated being a b-frame, and sending streamingmedia data with higher priority than b-frame to the client.
 56. A methodof claim 54, wherein said streaming media data being eliminated being ap-frames and b-frame, and sending frames with higher priority than bothp-frames and b-frames to the client.
 57. A method of frame thinning byclient comprising: sending a message to a caching proxy server, saidmessage indicating a need to thin streaming media data received at saidclient; receiving media back from said caching proxy server that arehigher in order than low order streaming media data.
 58. A method ofclaim 57, further comprising: sending a subsequent message from a clientto further thin streaming media data; receiving streaming media datathat is higher in order then previous streaming media data received. 59.A method of claim 57, further comprising: assigning an unsigned integerto a frame associated with streaming media data, wherein said assigningfurther comprising assigning an integer value of “0” to an unknown frametype; an integer value of “1” to a key frame type; an integer value of“2” to a p-frame type; and an integer value of “3” to a b-frame type.60. A method of claim 59, wherein said key-frame being most important inpriority than any other frames.
 61. A method of claim 59, wherein saidp-frame being less important in priority than key-frame, and moreimportant in priority than b-frame.
 62. A method of claim 59, whereinsaid b-frame being less important in priority than p-frame.
 63. A methodof claim 59, wherein said b-frame being less important in priority thankey-frame.
 64. A method of claim 59, wherein said unknown-frame beingeither more or less important in priority than key-frame, p-frame andb-frame.
 65. A method of claim 59, wherein said key-frame being moreimportant in priority than p-frames, b-frames, and any other frames. 66.A method of claim 58, wherein said sending comprising eliminatingp-frames and sending selected streaming media data that is higher inorder than p-frames to the client.
 67. A method of claim 58, whereinsaid sending comprising eliminating both p-frames and b-frames, andsending selected streaming media data that is higher in order than bothp-frames and b-frames.
 68. A method of using transmit time of RTP packettransmissions at a caching proxy server said method comprising:requesting data corresponding to transmit time for streaming media datafrom a server; receiving said streaming media data and correspondingtransmit time information from the server; storing the receivedinformation; and transmitting from said caching proxy server to a clientsaid streaming media data at times specified by said transmit time.69.-88. (canceled)
 89. A machine-readable medium that providesexecutable instructions, which when executed by a set of processors,cause said set of processors to perform RTP header extending operationscomprising: adding a first RTP sub-extension ID to an RTP header;defining a length of the first RTP sub-extension by providing asub-extension length; providing data corresponding to the RTPsub-extension ID within said length defined for said first RTPsub-extension; and having subsequent RTP sub-extensions following thefirst RTP sub-extension.
 90. A machine-readable medium as in claim 89,wherein said length of said RTP sub-extension being defined by a wholenumber of 32 bit words.
 91. A machine-readable medium as in claim 89,wherein said first RTP sub-extension immediately following the RTPheader.
 92. A machine-readable medium as in claim 89, wherein said RTPsub-extension length is immediately followed by RTP sub-extension dataand immediately preceded by RTP sub-extension ID.
 93. A machine-readablemedium as in claim 89, wherein said RTP sub-extension contains transmittime information of each RTP packet.
 94. A machine-readable medium as inclaim 89, wherein said RTP sub-extension contains persistent IDinformation.
 95. A machine-readable medium as in claim 89, wherein saidRTP sub-extension contains RTP Frame Type information.
 96. Amachine-readable medium as in claim 95, wherein said frame type being a16-bit unsigned integer value representing a different frame for eachvalue.
 97. A machine-readable medium as in claim 96, wherein saidunsigned integer and frame type further comprising steps of: assigningan integer value of “0” to an unknown frame type; an integer value of“1” to a key frame type; an integer value of “2” to a p-frame type; andan integer value of “3” to a b-frame type.
 98. A machine-readable mediumas in claim 97, wherein said key-frame being most important in prioritythan any other frames.
 99. A machine-readable medium as in claim 97,wherein said p-frame being less important in priority than key-frame,and more important in priority than b-frame.
 100. A machine-readablemedium as in claim 97, wherein said b-frame being less important inpriority than p-frame.
 101. A machine-readable medium as in claim 97,wherein said b-frame being less important in priority than key-frame.102. A machine-readable medium as in claim 97, wherein saidunknown-frame being either more or less important in priority thankey-frame, p-frame and b-frame.
 103. A machine-readable medium as inclaim 97, wherein said key-frame being more important in priority thanp-frames and b-frames other frames.
 104. A machine-readable medium thatprovides executable instructions, which when executed by a set ofprocessors, cause said set of processors to perform negotiatingoperations for various types of streaming media data by a servercomprising: receiving a request for one or more types of streaming mediadata from a caching proxy server or a client, said request including arequest for data associated with said streaming media data, said requestincluding an identifier which represents one of several possible typesof data associated with said streaming media data; determining ifrequested types of streaming media data are supported by the server; andresponding to the request with a response indicating the capability ofthe server to support the request.
 105. A machine-readable medium as inclaim 104, further comprising receiving a request to send supported RTPextensions to the caching proxy or the client.
 106. A machine-readablemedium as in claim 105, further comprising responding to request to sendand sending all the supported and requested extensions.
 107. Amachine-readable medium as in claim 104, further comprising receiving acommand terminating the negotiation process.
 108. A machine-readablemedium that provides executable instructions, which when executed by aset of processors, cause said set of processors to perform negotiatingoperations for various types of streaming media data by a caching proxyserver comprising: sending a request for one or more types of related orunrelated streaming media data to a server, said request including arequest for data associated with said streaming media data, said requestincluding an identifier which represents one of several possible typesof data associated with said streaming media data; receiving a responseto each requested type of streaming media data; and deciding whether toproceed or terminate negotiation process associated with streaming mediadata.
 109. A machine-readable medium as in claim 108, wherein decidingcomprises: determining if any requested type of streaming media data isnot supported by server; checking if unsupported type of streaming mediadata is essential to caching proxy server operations; and sending anexecution command to server.
 110. A machine-readable medium as in claim108, wherein determining supported types of streaming media data beingperformed by checking if a response in a form of an echo or otherwisehas been sent for requested type of streaming media data.
 111. Amachine-readable medium as in claim 109, wherein said execution commandbeing send based on results of checking if unsupported type of streamingmedia data is essential to caching proxy server operations.
 112. Amachine-readable medium as in claim 108, wherein said decision being toterminate negotiation process.
 113. A machine-readable medium as inclaim 108, wherein said decision being to proceed with negotiationprocess and requesting the server to send remaining supported type ofstreaming media data.
 114. A machine-readable medium that providesexecutable instructions, which when executed by a set of processors,cause said set of processors to perform frame thinning operations by acaching proxy server comprising: receiving a message from a client tothin frames in a streaming media data transmission from said cachingproxy server; evaluating priority of frames; and sending only selectedframes.
 115. A machine-readable medium as in claim 114, wherein saidframe priority being evaluated by naming unsigned integers to frametypes, said unsigned integers comprising: assigning an integer value of“0” to an unknown frame type; an integer value of “1” to a key frametype; an integer value of “2” to a p-frame type; and an integer value of“3” to a b-frame type.
 116. A machine-readable medium as in claim 114,wherein said key-frame being most important in priority than any otherframes.
 117. A machine-readable medium as in claim 114, wherein saidp-frame being less important in priority than key-frame, and moreimportant in priority than b-frame.
 118. A machine-readable medium as inclaim 114, wherein said b-frame being less important in priority thanp-frame.
 119. A machine-readable medium as in claim 114, wherein saidb-frame being less important in priority than key-frame.
 120. Amachine-readable medium as in claim 114, wherein said unknown-framebeing either more or less important in priority than key-frame, p-framea b-frame.
 121. A machine-readable medium as in claim 114, wherein saidkey-frame being more important in priority than p-frames, b-frames, andany other frames.
 122. A machine-readable medium as in claim 114,further comprising: receiving a second request from a client to furtherthin frames; processing request and eliminating more selected frames andsending frames of higher priority.
 123. A machine-readable medium as inclaim 122, wherein said selected frame being eliminated being a p-frame,and sending frames with higher priority than p-frame to the client. 124.A machine-readable medium as in claim 122, wherein said selected framebeing eliminated being a p-frames and b-frame, and sending frames withhigher priority than both p-frames and b-frames to the client.
 125. Amachine-readable medium that provides executable instructions, whichwhen executed by a set of processors, cause said set of processors toperform frame thinning operations by a client comprising: sending athinning message to a caching proxy server indicating not to send loworder frames; receiving frames back from caching proxy server that arehigher in order than low order frames.
 126. A machine-readable medium asin claim 125, further comprising: sending a subsequent message from aclient to further thin frames; receiving frames that are higher in orderthen previous frames received.
 127. A machine-readable medium as inclaim 125, wherein said frames being assigned unsigned integers, saidunsigned integers comprising: assigning an integer value of “0” to anunknown frame type; an integer value of “1” to a key frame type; aninteger value of “2” to a p-frame type; and an integer value of “3” to ab-frame type.
 128. A machine-readable medium as in claim 125, whereinsaid key-frame being most important in priority than any other frames.129. A machine-readable medium as in claim 125, wherein said p-framebeing less important in priority than key-frame, and more important inpriority than b-frame.
 130. A machine-readable medium as in claim 125,wherein said b-frame being less important in priority than p-frame. 131.A machine-readable medium as in claim 125, wherein said b-frame beingless important in priority than key-frame.
 132. A machine-readablemedium as in claim 125, wherein said unknown-frame being either more orless important in priority than key-frame, p-frame and b-frame.
 133. Amachine-readable medium as in claim 125, wherein said key-frame beingmore important in priority than p-frames, b-frames and any other frames.134. A machine-readable medium as in claim 126, wherein said eliminatingcomprising eliminating p-frames and sending selected frames that arehigher in order than p-frames to the client.
 135. A machine-readablemedium as in claim 126, wherein said eliminating comprising eliminatingboth p-frames and b-frames, and sending selected frames that are higherin order than both p-frames and b-frames.
 136. A machine-readable mediumthat provides executable instructions, which when executed by a set ofprocessors, cause said set of processors to send transmit time of RTPpacket transmission operations from a caching proxy comprising:requesting data corresponding to transmit time for streaming media datafrom a server; receiving said streaming media data corresponding withtransmit time information from the server; storing the receivedinformation; and transmitting from said caching proxy server to aclient, said streaming media data at times specified by said transmittime. 137.-139. (canceled)
 140. A RTP header comprising: means foradding a first RTP sub-extension ID to an RTP header; means for defininga length of said first RTP sub-extension by providing a sub-extensionlength; means for providing data corresponding to the RTP sub-extensionID within said length defined for said first RTP sub-extension; andmeans for having subsequent RTP sub-extensions following the first RTPsub-extension. 141.-142. (canceled)
 143. A frame thinning caching proxyserver comprising: means for receiving a message from a client to thinframes in a streaming media data transmission from said caching proxyserver; means for evaluating priority of frames; and means for sendingonly selected frames.
 144. A client comprising: means for sending amessage to a caching proxy server, said message indicating a need tothin streaming media data received at said client; means for receivingmedia back from caching proxy server that are higher in order than loworder media.
 145. (canceled)